>T AVAILABLE COPY 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
H04N 7/173 



Al 



(11) International Publication Number: WO 98/18260 

(43) International Publication Date: 30 April 1998 (30.04.98) 



(21) International Application Number: PCT/US97/ 19448 

(22) International Filing Date: 24 October 1997 (24.10.97) 



(30) Priority Data: 
08/738,361 



25 October 1996 (25.10.96) 



US 



(71) Applicant: DIVA SYSTEMS CORPORATION [US/US]; 

Building 203, 333 Ravenswood Avenue, Menlo Park, CA 
94025 (US). 

(72) Inventors: GOODE, Christopher; 722 Creek Drive, Menlo 

Park, CA 94025 (US). GORDON, Donald; 216 Locust 
Street, Pacific Grove, CA 93950 (US). CONOVER, Mark, 
D.; 10721 Wunderlich Drive, Cupertino, CA 95014 (US). 
ILLOWSKY, Daniel; 363 Dexter Drive, Cupertino, CA 
95014 (US). THOMAS, Philip, A.; 1410 Cedarmeadow 
Court, San Jose, C A 95131 (US). COLE, Brooks; 65 Third 
Street, Point Reyes Station, CA 94956 (US). 

(74) Agent: MOSER, Raymond, R., Jr.; Thomason & Moser, 2-40 
Bridge Avenue, P.O. Box 8160, Red Bank, NJ 07701 (US). 



(81) Designated States: AU, CA, CN, JP, KR, MX, SG, European 
patent (AT, BE, CH, DE, DK, ES, FI, FR, GB, GR, IE, IT, 
LU, MC, NL, PT, SE). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: METHOD AND APPARATUS FOR MASKING THE EFFECTS OF LATENCY IN AN INTERACTIVE INFORMATION 
DISTRIBUTION SYSTEM 
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A method (200) and apparatus for masking the effects of la- 
tency within an information distribution system (100). The apparatus 
comprises a set top terminal (108) that requests and receives informa- 
tion from an information server (102). The information is generally 
displayed upon a conventional television (110) coupled to the set top 
terminal which contains a central processing unit (122) and an infor- 
mation stream decoder (130) programmed to implement the method. 
Upon execution of certain functions selected by a subscriber usually 
via a remote control (112), the method recalls (205) a predefined im- 
age from memory and begins to fade (206) the presently displayed 
image into a predefined image, e.g., a white screen. Simultaneously, 
the presently playing sound is also faded into the predefined sound, 
e.g., no sound. The decoder within the set top is reset (210). Lasdy, 
the set top terminal fades (214) up from the predefined image to the 
new decoded video stream and from the predefined audio to the new 
audio stream. 
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METHOD AND APPARATUS FOR MASKING THE EFFECTS 
OF LATENCY IN AN INTERACTIVE 
INFORMATION DISTRIBUTION SYSTEM 

5 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to an interactive information distribution 
10 system such as a video-on-demand (VOD) system. More particularly, the 
present invention relates to a method and apparatus for masking the effects 
of latency within such an interactive information distribution system. 

2. Description of the Background Art 

15 Recent advances in digital signal processing techniques and, in 

particular, improvements in digital compression techniques, have led to a 
plethora of proposals for providing new digital services to a subscriber's home 
via existing telephone and coaxial cable networks. For example, it has been 
proposed to provide hundreds of cable television channels to subscribers by 

20 compressing digital video, transmitting the compressed digital video over 
conventional coaxial cable television channels, and then decompressing the 
video at a subscriber's set top terminal. Another proposed application of this 
technology is a video-on-demand system in which a subscriber communicates 
directly with a video service provider via telephone lines to request a video 

25 program from a video library and the requested video program is routed to the 
subscriber's home via telephone lines or via coaxial cable television cables for 
immediate viewing. Other proposed video-on-demand systems use a 
frequency multiplexing technique to enable control information from a set top 
terminal to be transmitted through a cable network back to an information 

30 server. Such a system permits bi-directional communications between the 
service provider and the subscriber over a single network. 

In each of these information distribution systems, the compression and 
decompression techniques as well as processing of control information 
requires a substantial amount of time. As such, a customer experiences a 

35 significant delay or latency between the selection of a function via their set 
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top terminal and when the selected function is actually implemented upon the 
screen of their television. Heretofore this latency has merely been an 
annoyance which, for the most part, has been ignored by the interactive 
information distribution system providers. Typically, upon selection of a 
5 certain function, such as stop or pause, while viewing a particular video 
stream, the present systems merely delay the time of implementing that 
stop or pause function by the amount of latency. The latency can be in 
excess of five seconds. Furthermore, if a customer has a particular video 
stream being displayed and then selects a second video stream for display, 

10 present systems, after a slight control signal processing delay, switch from 
one video stream to another video stream without any attempt to smooth the 
transition. As such, the video that is received and decoded by the user's set 
top terminal is generally garbled or distorted during the transition period. If a 
customer is rapidly switching from video stream to video stream, e.g., channel 

15 surfing, the customer will view garbled video images for a substantial amount 
of time between "channel" changes. Such an extended period of garbled 
images is unacceptable for a commercial system. 

Therefore, there is a need in the art for a method and apparatus for 
masking the effects of latency in an interactive information distribution 

20 system. 

SUMMARY OF THE INVENTION 
The disadvantages heretofore associated with the prior art are 
overcome by the present invention of a method and apparatus for masking 

25 the effects of latency within an interactive information distribution system. 
The apparatus of the present invention comprises a set top terminal for 
requesting and receiving information from an information server. The 
requested information is typically displayed upon a conventional television 
that is connected to the set top terminal. The set top terminal contains a 

30 central processing unit and a compressed video decoder that are programmed 
to implement the method of the present invention. The method is routine 
which is executed when a subscriber selects certain functions, usually via an 
input device such as a remote control, that are used to control the flow of 
information to a television in the subscriber's home. These certain functions 
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that result in execution of the latency masking routine are those that require 
the information server to provide a new information stream, e.g., selecting a 
new program for viewing, changing on-screen menus, and the like. 

Once executed, the routine first acknowledges the selected function by 
5 displaying the function name or icon as a graphic on the screen of the 

television. This graphic is generated by the set top terminal. Simultaneous 
with generating an acknowledgment, the set top terminal sends an upstream 
command to implement the selected function to the information server via a 
command channel of the information distribution system, e.g., a telephone 

10 network, a frequency multiplex channel through a cable network, and the like. 
Thereafter, the method begins to fade the presently displayed image into a 
predefined image, e.g., a white screen, a graphic image, animation, a video clip, 
and the like. Simultaneously, the presently playing audio is also faded to a 
predefined audio, e.g., no sound or a sound that compliments the predefined 

15 image. The decoder within the set top terminal is reset to flush from its buffer 
any residual audio or video data from the previously decoded information 
stream. Thereafter, the set top terminal awaits a new information stream to 
arrive from the information server while presenting the customer with the 
predefined image and audio. Lastly, once the new information stream has 

20 arrived and begins to be decoded, the set top terminal fades from the 
predefined image to the new information stream and from the pre-defined 
audio to the audio stream that accompanies the new video stream. At this 
point, the customer's selected function has been fully implemented without 
viewing the detrimental affects usually associated with a latency delay, i.e., 

25 the latency has been masked. Once complete, the latency routine becomes 
dormant and awaits the next function to be selected by the customer. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The teachings of the present invention can be readily understood by 
30 considering the following detailed description in conjunction with the 
accompanying drawings, in which: 

FIG. 1 depicts a high level block diagram of an interactive information 
distribution system containing the present invention; and 
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FIG. 2 depicts a flow chart of a software implementation of the present 
invention. 

To facilitate understanding, identical reference numerals have been 
used, where possible, to designate identical elements that are common to the 
5 figures. 

DETAILED DESCRIPTION 
FIG. 1 depicts a high level block diagram of an information distribution 
system 100 that incorporates the present invention. The system contains an 

10 information server 102, a session manager 104, a communications network 
106, set top terminal 108, a display device 110, and an input device 112 such 
as a remote control for inputting functions into the set top terminal 108. The 
information server 102 provides data streams in response to requests for 
information from the session manager 104. These data streams are 

15 packetized and modulated onto a carrier that is compatible with the 
transmission requirements of the network 106. The data streams are 
propagated to the set top terminal 108 via a first forward channel, e.g., 
having a carrier frequency of 50 to 750 MHz and a bandwidth of 
approximately 6 MHz. Additionally, the system uses a second forward 

20 channel for communicating command and control information from the server 
to the set top terminal, e.g., having a carrier frequency of 50 to 750 MHz and 
a bandwidth of 1 MHz. Also, the system uses a back channel (or reverse 
channel) for communicating command and control information from the set 
top terminal 108 to the server 102. The back channel has a carrier frequency 

25 of 5-42 MHz and a bandwidth of 250 kHz. Although the invention is described 
with respect to this three channel, interactive data distribution system, it 
should be understood that the invention will function in the context of 
broadcast-only systems (e.g., systems having a single forward channel) as 
well as any system that experiences latency before implementing a 

30 command. 

The session manager 104 handles all of the transmission interface 
requirements of the system 100. The network can be any one of a number of 
conventional communications networks that are available such as a hybrid 
fiber-coax network, telephone network, existing cable television network and 
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the like. For example, if the network is a hybrid fiber-coax network, the 
transmission transport technique used may be modeled after the Moving 
Pictures Experts Group (MPEG) transport protocol to transport the 
information streams from the server to the set top terminals. In general, the 
transport mechanism for the first forward channel that transports 
broadband information to the set top terminal must be able to carry 
unidirectional, asynchronous packetized data, such as that defined in the 
asynchronous transfer mode (ATM) standard, the Moving Pictures Experts 
Group (MPEG) standard, and the like. There are many such broadband 
forward channel transport mechanisms available. 

The set top terminal 108 receives the data streams from the forward 
channel, demodulates those streams and processes them for display on the 
display device 110 (e.g., a conventional television). In addition, the set top 
terminal 108 accepts commands from a remote control 112 or other input 
device. These commands are formatted, compressed, modulated, and 
transmitted through the network to the session manager using the back 
channel. These commands may be transmitted through the same network 
used to transmit information to the set top terminal. However, the back 
channel from the set top terminal to the server may be a separate network, 
e.g., the first forward channel is through a television cable network and the 
back channel and the second forward channel are through a telephone 
network. The session manager interprets the commands sent from the set 
top terminal through the back channel and instructs the information server 
to perform the command. 

More particularly, the information server 102 contains at least one 
central processing unit 114 and, in a parallel processing computer based 
server, contains more than one central processing unit 114. Additionally, the 
server generally contains a program memory 116 that retains the programs 
which control the server. Some of these programs are executed in response to 
function requests sent from set top terminals. In addition, the CPU is 
connected to a data memory 118 that contains the various video and other 
information that is available to the customers via the system 100. 

Additionally, the set top terminal 108 contains a receiver/transmitter 
(transceiver) 120 for receiving information from both forward channels and 
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transmitting commands through the back channel. If the forward and back 
channels use a common network, the transceiver is generally operated in a 
frequency multiplex manner. The transceiver 120 downconverts and 
demodulates the signals from the network using a conventional 
5 superhetrodyne downconversion technique, a conventional QAM demodulator 
for the first forward channel and a conventional QPSK demodulator for the 
second forward channel. The first forward channel signals are generally 
transmitted in a standard packetized format such as the MPEG transport 
protocol of a transport stream. The transceiver contains a conventional 

10 MPEG transport decoder to extract particular program streams from the 
transport stream. 

Any video data carried within the program streams via the forward 
channel is generally compressed using either MPEG-1 or MPEG-2 
compression. The compressed video is processed using a conventional MPEG 

15 program stream decoder such as the model L64002 manufactured by LSI 
Logic of Milpitas, California. This decoder, when provide with an MPEG-2 
compliant bit stream (program stream(s)) produces digital audio and digital 
video signals. These signals are further processed to convert the digital 
signals into composite video or RGB signals. To view the video signals on a 

20 conventional television, the composite video signal is modulated onto a 
channel 3/4 carrier and coupled to the display 110. 

The set top terminal processes the decompressed information for 
display upon the display unit 110. The process for converting the 
decompressed data into information that is displayed is handled, for the most 

25 part, by the decoder 130. Command and control functions are accomplished 
by a central processing unit (CPU)122 operating in conjunction with well 
known support circuits 124 such as power supplies, cache, clock circuits and 
the like. In addition, the CPU is coupled to a program memory 126 that 
stores the programs which are executable by the set top terminal CPU and 

30 image memory 128 that contains a plurality of bit map images. The bit map 
images can be selectively displayed upon the display unit on an as needed 
basis. Depending upon the size of the image memory, it may also store video 
clips, audio clips, animation, graphical images, and the like. The present 
invention is implemented as a software program that is stored in the program 
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memory 126 and, when executed on the central processing unit 122, uses 
predefined images that are stored in the image memory 128 as well as control 
decoder processing to "freeze" certain decoded imagery on the display. The 
software implementation is a latency masking routine that is discussed in 
5 detail with reference to FIG 2. 

FIG. 2 depicts a flow chart of the latency masking routine 200 of the 
present invention that is implemented in software and executed by the CPU 
within the set top terminal. This latency masking routine is executed 
whenever a customer selects a particular function that will result in latency, 

10 e.g., any function that must be implemented by the server such as a function 
that affects the data stream. Once such a function is selected, the routine 
200 is executed beginning at step 202 and continuing to step 204. At step 
204, the routine acknowledges the selected function by displaying that 
particular function by name or icon upon the screen of the display device. 

15 Display of such function information is generally handled by recalling a 

particular bit map image, overlay image, or on-screen display (OSD) graphic 
from the image memory for display upon the display unit. Generally, this 
display is accomplished in an overlay manner so that the function appears 
semi-transparent or, possibly, fully opaque in, for example, the upper left 

20 hand corner of the presently displayed information. The simplest method for 
generating such a graphic is to use the OSD function of the MPEG decoder to 
produce a character set on the display unit. The process used to generate 
OSD characters using an MPEG decoder are well-known in the art. 

At the same time as the routine begins acknowledging the selected 

25 function, the routine, at step 208, instructs the transceiver to send the 
upstream command to the information server. In response, the session 
manager processes the command and sends the command to the server. As 
a result of the command, a new video stream will be sent to the set top 
terminal. This new video may be an additional menu, a newly selected movie, 

30 and the like. 

At step 205, the routine generates a predefined image. This image 
may be recalled from the image memory (128 in FIG. 1) and sets a transition 
timer within the decoder. As shall be described below, when the transition 
timer expires (times out), the routine begins a fade to the new video sequence. 
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The predefined image can be a bit map image, but is more typically, a 
previously received and decoded sequence of MPEG encoded packets. In the 
preferred embodiment of the invention, the CPU instructs the MPEG decoder 
to "freeze" the image that is presently being displayed. Hius, the predefined 
5 image is all or a portion of a frozen last frame of video. 

At step 206, the routine begins fading the presently displayed video 
image into the predefined image. In addition, the presently playing audio is 
also faded to a predefined audio signal, e.g., no sound. Typically, if predefined 
audio is desired, the predefined audio compliments or is, in some manner, 

10 associated with the predefined image stored in the image memory. 

Preferably, the predefined image is a white background containing a centrally 
located geometric shape or object. The object is generally designed to distract 
the user or the customer from observing the latency during command 
implementation. Hie predefined image is not required to be static and, for 

15 example, can be a video clip, animation segment, a series of static images and 
the like. 

In the preferred embodiment, the frozen image contains a centrally 
located object that is common to the beginning and ending frames of all video 
images displayed by the system. To facilitate a transition from one video 

20 segment to another, the OSD function of the MPEG decoder is used to "fade" 
all or part of the frozen image to white. However, the centrally located object 
is not faded. Hie fade is accomplished by designating the entire screen except 
for the object as an OSD graphic region and then fading the graphic overlay 
from transparent to opaque. The centrally located object is keyed or other 

25 wise associated with the previous video segment and is also keyed or 
associated with the upcoming video segment As such, this object is 
contained in the beginning and ending sequence of all displayed video and the 
viewer experiences a spacial continuity from video segment to video segment. 
While the decoder is frozen (or decoding the latency masking imagery), 

30 the routine causes the set top terminal to reset the decoder contained in the 
set top terminal receiver such that the previous images that were being 
decoded are flushed from the buffer memory of the decoder. Of course, the 
frozen frame of video information is not flushed from the memory. Moreover, 
at step 210, other housekeeping measures may be accomplished to clean up 
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registers and other memory devices that were handling the decoding of the 
previous information stream. At step 212, once the decoder is reset, the set 
top terminal awaits the new information stream while continuously 
displaying the predefined image. At step 214, the new information stream is 
5 received and begins to be decoded such that the set top terminal may then 
fade from the predefined image that is presently being displayed to imagery 
decoded from the new information stream. Also, the predefined audio is faded 
to the new audio that accompanies the new information stream. Generally, 
the time required to fade up and fade down the predefined image is 

10 approximately two seconds. Of course, this time is directly related to the 
expected latency of the information distribution system and is typically 
established using a transition timer within the decoder. The cropped frozen 
image is repeatedly refreshed on display until the transition timer times out. 
To establish the spacial continuity, the decoder is "unfrozen" to 

15 produce the new video information which begins with a sequence having the 
centrally located object. The OSD graphics remain opaque to crop all imagery 
but the object. Then the OSD graphics fade to transparent to reveal the new 
video image. As such, the new video information may contain a "precursor" 
video segment that displays additional transitional imagery prior to the new 

20 video information (e.g. , movie) beginning. 

At step 216, the latency masking routine ends and awaits the next 
function selected by the subscriber. 

By using the method and apparatus of the present invention any 
latency within the system is effectively masked by the latency masking 

25 routine such that the customer does not experience any annoying artifacts 
displayed on the screen while transitioning from one video sequence to 
another. In addition, the predefined image that is displayed is visually 
pleasing such that the latency is not noticed by the customer. Furthermore, 
the latency mask provides spatial continuity between image sequences such 

30 that a viewer is focused on the images and not the latency. 

Although the foregoing description of the invention focused upon use of 
the invention in a video-on-demand system, the invention finds use in other 
information distribution systems that experience latency. For example, an 
INTERNET browser could execute the latency masking routine while 
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retrieving a new page of information, downloading lengthy documents, or 
accomplishing a search. In general, the present invention finds use wherever 
system latency is observed in an interactive information distribution system. 

Although one embodiment which incorporates the teachings of the 
present invention has been shown and described in detail herein, those skilled 
in the art can readily devise many other varied embodiments that still 
incorporate these teachings. 
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What is claimed is: 

1 . In a set top terminal ( 108) used for decoding and displaying imagery 

5 associated with information streams transmitted to said set top terminal via 
an information distribution system (100), a method (200) for masking latency 
within said information distribution system, where said latency occurs while 
transitioning from a first information stream to a second information stream, 
said method comprising the steps of : 
10 responding (204) to a user command entered into said set top terminal 

and, simultaneously, sending (208) an upstream data command to the 
information distribution system to implement said user command; 

recalling (205) a predefined image from an image memory (128); 

fading (206) at least a portion of imagery decoded from said first 
15 information stream into said predefined image; 

resetting (210) a signal decoder (130) within said set top terminal; 

fading (214) from said predefined image into imagery decoded from said 
second information stream; and 

displaying (214) said imagery from said second information stream. 

20 

2. The method of claim 1 further comprising the steps of: 

fading (206) an audio signal associated with said imagery of said first 
information stream into a predefined audio signal as said imagery of said first 
information stream is faded into said predefined image; and 
25 fading (2 14) from said predefined audio signal into an audio signal 

associated with said second information stream as said predefined image is 
faded into said imagery of said second information stream. 

3. The method of claim 1 wherein said predefined image is a white screen, 
30 a white screen containing at least one geometric object, animation, or a video 

clip. 

4. The method of claim 3 wherein said predefined audio signal produces no 
sound or a sound that compliments the predefined image. 
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5. The method of claim 1 further comprising the steps of: 
displaying (205), in response to said user command, a function 

selection indicator that identifies a function that corresponds to the user 
5 command. 

6. The method of claim 1 wherein said predefined image is a portion of a frozen 
video frame. 

10 7. In a set top terminal (108) used for decoding and displaying imagery 

associated with information streams transmitted to said set top terminal via 
an information distribution system (100), a method (200) for masking latency 
within said information distribution system, where said latency occurs while 
transitioning from a first information stream to a second information stream, 
15 said method comprising the steps of : 

responding (204) to a user command entered into said set top terminal 
and, simultaneously, sending (208) an upstream data command to the 
information distribution system to implement said user command; 

freezing (205) an image frame of said first information stream and, 
20 simultaneously, activating a transition timer; 

fading (206) at least a portion of said frozen image frame by masking 
said portion using an on-screen display overlay; 

resetting (210) a signal decoder within said set top terminal; 
upon time out of the transition timer, fading (214) from said predefined 
25 image into imagery decoded from said second information stream; and 

displaying (214) said imagery from said second information stream. 

8. The method of claim 7 wherein said on-screen display overlay is a white 
screen that covers at least a portion of said frozen image frame. 

30 

9. The method of claim 7 further comprising the steps of: 

displaying (214), in response to said user command, a function 
selection indicator that identifies a function that corresponds to the user 
command. 
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10. The method of claim 7 wherein said on-screen display overlay covers all 
but an object in said frozen image frame and the object is contained in the 

; second information stream, where having said object in both sad first 

5 information stream to provide said frozen image frame containing the object 
and said second information stream provides spacial continuity from the first 
information stream to the second information stream. 

11. In a set top terminal (108) used for decoding and displaying imagery 

10 associated with information streams transmitted to said set top terminal via 
an information distribution system (100), apparatus for masking latency 
within said information distribution system, where said latency occurs while 
transitioning from a first information stream to a second information stream, 
said apparatus comprising: 

15 means (204, 208) for responding to a user command entered into said 

set top terminal and for simultaneously sending an upstream data command 
to the information distribution system to implement said user command; 

memory access means (205) for recalling a predefined image from an 
image memory (128) in response to said user command being entered; 

20 first image processing means (206) for fading at least a portion of 

imagery decoded from said first information stream into said predefined 
image; 

controller means (122) for resetting (210) a signal decoder (130) within 
said set top terminal; 
25 second image processing means (214) for fading from said predefined 

image into imagery decoded from said second information stream; and 

means (214) for displaying said imagery of said second information 
stream. 

30 12. The apparatus of claim 11 further comprising: 

means (206), coupled to said first image processing means, for fading 
an audio signal associated with said imagery of said first information stream 
into a predefined audio signal as said imagery of said first information stream 
is faded into said predefined image; and 
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means (214), coupled to said second image processing means, for fading 
from said predefined audio signal into an audio signal associated with said 
second information stream as said predefined image is faded into said imagery 
of said second information stream. 

5 

13. The apparatus of claim 1 1 wherein said predefined image is a white 
screen, a white screen containing at least one geometric object, animation, or 
a video clip. 

10 14. The apparatus of claim 12 wherein said predefined audio signal produces 
no sound or a sound that compliments the predefined image. 

15. The apparatus of claim 11 further comprising: 

means (205) for displaying, in response to said user command, a 
15 function selection indicator that identifies a function that corresponds to the 
user command. 

16. The apparatus of claim 11 wherein said predefined image is a portion of a 
frozen video frame. 

20 

17. The apparatus of claim 16 wherein said predefined image contains an 
on-screen display overlay that covers all but an object in said frozen image 
frame and the object is also contained in the second information stream, 
where having said object in both sad first information stream to provide said 

25 frozen image frame containing the object and said second information stream 
provides spatial continuity from the first information stream to the second 
information stream. 
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